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INTRODUCTION 



This compendium contains a classification of the author-generated 
"suggestions for additional research" from the literature which was 
abstracted under Air Force Contract No* F336l5'-72'-C-l88A . These research 
and devrlopment needs were initially categorized under the nine major ISD 
activities listed in the Handbook for Designers of Instructional Systems^ . 
Each of these nine major categories was then further subdivided, if the 
number and nature of the research suggestions permitted more specific 
classifications. Following is a brief description of the major ISD acti- 
vities : 



!• Detemine Job Performance Requirements (JPRs ) ♦ The process 
of determining the tasks required of the human component, 
and the standards of performance. This process applies to 
all types of "jobs." It results in a statement of all 
human activities (skills, knowledges, and attitudes) re- 
quired for successful performance. 

2. Determine Training Requirements (TRs ) . The process of 
determining the changes needed in skills, knowledges, and 
attitudes of personnel, so they can perform a job. These 
changes, when added to the entering repertoire of abili- 
ties, must meet the JPRs. 

3» Determine Criterion Objectives * The process of specifying 
the objectives which the student must meet to satisfy the 
TRs. Criterion objectives specify precisely what behavior 
is to be exhibited, the conditions under which behavior wil 
will be accomplished, and the minimum standard of accept- 
able performance. 

A. Develop Criterion-Referenced Tests * The process of develop- 
ing and administering tests which directly measure the 
criterion objectives. The survey test is administered to 
samples of prospective students . The purpose is to verify 
which skills and knowledges to include in the course of 
instruction. Criterion-referenced tests (course criterion 
tests and diagnostic tests) are also developed to determine 
if the behaviors in the criterion objectives have been 
acquired . 



'Department of the Air Force. Handbook for designers of instructional 
systems . Washington, D.C.: June 1973. AFP 50-58. 
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S elect Media/Methods , The process of selecting appropriate 
media and methods for each block of instructional objec- 
tives. Selection is based on: 



a. Practical constraints (such as financial consider- 
ations) 



b. Instructional nature of the objectives (certain 
behaviors may be Important In training, but not 
on the job) 

c. Presentation mode implied by the objectives 
(visual , auditory , etc . ) 

d. Type of learning involved (for example, simple 
visual discrimination; chain of skilled perform- 
ances) 



e. Best instructional sequence for the objectives. 

6. Develop Instructional Materials . The process of developing 
and integrating the actual materials which make up the 
instructional regimen . 

7. Validate and Revise Instructional Materials . The process 
by which each unit of instruction is tested (validated) 
as it is developed. This process insures that criterion 
objectives are satisfied. First, materials are tested on 
several individuals and revised as necessary. Then, they 
are tried out on small groups of students, carefully 
sampled from the potential student population. Final revi- 
sions are made. 



8. Conduct Instructional Program . The process of implementing 
and administering the instructional program. This includes 
training of instructors and scheduling as well as the actual 
conduct of the program. 

9. Evaluate Instructional Program . The process of determining 
the extent to which graduates of the Instructional program 
satisfy the performance requirements in the job environment. 
Detailed records of graduate performance are kept, and 
changes to the instructional program are recommended as 
necessary. 



The major category "Conduct Instructional Program" has been omitted 
from this classification scheme* After reviewing all of the research 
suggestions and classlEicatlon sections, it appeared that none of the 
research needs directly related to that topic; rather most were indirectly 
applicable. Instead of classifying all Indirectly related suggestions, a 
decision was made to omit this category entirely and to categorize the 
suggestions in the areas where they were more directly relevant. Also, 



for the same reasons, research needs which might have been classified in 
the major category, ^'Validate and Revise Instructional Materials/' have 
been listed under the appropriate subheadings of the major category, *'Evalu- 
ate Instructional Program," 

Within each category, the research needs are listed chronologically 
(according to publication date). Whenever a research suggestion was judged 
to be equally applicable to more than one category, the suggestion appears 
under each applicable category. The number in parentheses at the conclusion 
of each item are the bibliographic file sequence number and the year of 
publication. These references are provided because although many of the 
listed suggestions can be understood as presented in the compendium, others 
are difficult to interpret outside of the context of the original document. 
It is suggested, therefore, that the user who intends to follow up on a 
suggested research item, return to the original document (or its abstract) 
to determine the precise context of the recommendation. The complete 
bibliography of titles from which this compendium was drawn is presented 
in a companion document. 



Schumacher, S. P., Pearlstein, R. B., & Martin, P. W. A comprehensive key 
wovd index and bibliography on instructional system development. Dayton, 
Ohio: Air Force Human Resources Laboratory, Wright-Patterson Air Force 
Base, February 1974, in press. AFHRL-TR- 7A- 1 4 . 
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DETERMINE JOB PERFORMANCE REQUIREMENTS (JPRs) 



The following research suggestions are of use to the performance 
analyst who must carefully observe, analyze and classify those activities 
which are required to perform the task. It is suggested that research 
be conducted not only on present JPRs (task descriptions, performance re- 
quirements and task classification) , but also on future performance 
requirements that may result fro:n technological advances. 



A. Task Descriptions 



Research into the nature of abilities required for tracking tasks, or other 
tasks of similar complexity, (2309/1961) 

Research into multidimensionality and changes over time, which were felt 
to be two characteristics of job performance, criteria needing further 
attention and investigation in connection with matters of criterion 
reliability and predictability. (348/1963) 

Research to establish reliability of the task analysis method, (176/1964) 

Research in long-range improvements in Task Analysis and Training Situational 
Analysis, specifically "experimental work on characteristics of per- 
formance and on relationships between task attributes and training re- 
quirements. The immediate need is for integration and systematic tryout 
and testing of the present draining Situation Analysis techniques." 
(176/1964) 

Research to provide a sound foundation In the experimental fmalysis of 
human academic learning. (976/1972) 



B. Performance Requirements 



Analysis of the interrelationships among component performance measures of 
other complex jobs, to possibly provide one way of defining the ability 
requirements underlying proficiency in those jobs, (2269/1960) 

Further research into the nature of abilities required for tracking tasks 
or other tasks of similar complexity. ,(2309/1961) 

Research attempting to relate ability factors (as described by Fleisliman, 
1964, 1965b) to performance on the task factors identified in this 
study (Locke, Zav^-ila, and Fleishman, 1965). (1089/1965) 
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Studies to define training effects as a function of differing methods, 
media, and schedules applicable to the training system. (512/1966) 

More data collection on human performance in tasks and situations highly 
representative of the pilot's job under conditions found in flying. 
(512/1966) 

Regular, periodic evaluation of the relevance of technical curricula to 
the educational input into the labor market. (llOA/1970) 



C. Classification of Tasks 



Continuation of methodological research to extend the basic capabilities 
of job-standard definition; investigation of required allocation tech- 
niques and devising of an Index Of Task Accomplished (IOTA) describing 
stable characteristics of a system, including man; and integration of 
maintenance considerations into the job standards method. (219/1964) 

Research on how the cognitive and effective domain of the taxonomy of 

educational objectives and the classification of capabilities by Gagrie 
are related. (2186/1965) 

Development of an empirically grounded and logically sound taxonomy covering 
a wide range of learning situations. (485/1966) 

Research into these areas: (1) Refinement of the task classification 

system, (2) experimentation to determine reliability, consistency, and 
effectiveness of strategies prescribed for different task types, and 
(c) final test of the taxonoiTiy of response processes by practical 
application. (24/1969) 

Research on the task and cognitive variables which are relevant to per- 
formance, and ways of manipulating these variables. (2236/1969) 



D. Predicting Future Performance Requirements 

Research into multidimensionality and changes over time, which were felt 
to be two characteristics of job performance criteria needing further 
attention and investigation in connection with matters of criterion 
reliability and predictability. (348/1963) 

Research to develop more refined forecasting techniques, especially those 
used to identify the impact of technological changes on skill require- 
ments and demand for labor. (1104/1970) 
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DETERMINE TRAINING REQUIRE>tENTS (TRs) 



The following research needs suggest methods for determining the 
needed personnel skills, knowledges, and attitudes that are prerequisite 
to performing a job. 



Further research on whether a test of ability to follov; instructions has 
value in predicting a significant part of a particular criterion com- 
plex. (988/1961) 

Research in long-range improvements in Task Analysis and Training Situ- 
ational Analysis, specifically "experimental work on characteristics 
of performance and on relationships between task attributes and training 
requirements. The immediate need is for integration and systematic 
tryout and testing of the present Training Situation Analysis techniques." 
(176/1964) 

Development of a single intercorrelation matrix Including all qualitative 
variables available for prediction of student success in flight training 
to determine their total contribution to the multiple prediction formulae. 
(1332/1966) 

Preparation of an inventory of organized skill development programs carried 
out by formal institutions, the military, trade unions, etc* (1810/1966) 
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DETERl^lNE CRITERION OBJECTIVES 



Although the topic is widely discussed ^ and procedures often given, 
few research needs were found in this area. 

Research on how the cognitive and affective domain of the taxonomy of 

educational objectives and the classification of capabilities by Gagne 
are related. (2186/1965) 

Research to identify desirable changes in student behavior, and determination 
of the effects of new question strategies. Researchers should give "high 
priority" to these tasks. (1110/1970) 

The author recommended working toward a situation in which Instructional 
objectives and student characteristics determine the instruction 
strategy, which, in turn, determines the procedures for implementation 
of learning exercises. (1293/1970) 

Research to explore ways to maximize the effects on learning of complex 
combinations of variables. Including objectives* (2165/1972) 



DEVELOP CRITERION-REFERENCED TESTS 



Research is needed in the areas of test construction and validation. 
Research is also suggested to determine related effects of testing. 



A. Test Construction 



Experimentation with alphabetical item arrangement, and arrangement accord- 
ing to the printed length of the option of multiple-choice items. 
(947/1952) 

Development of a code aptitude test emphasizing the factors of Dot Esti- 
mation and End-element Substitution, since these two factors contribute 
the most errors. (2205/1955) 

Additional studies of the true-false formula using various kinds of subject 
matter and items at various levels of difficulty. Tape recordings of 
the verbalized Introspections of a group of examiners might serve as a 
means of obtaining data. (2259/1959) 

Research to determine whether there is a relationship between organization 
and better recall on a free recall test. (2560/1962) 

« 

Experimentation with reading comprehension measures that employ formats 
other than multiple-choice questions. (1781/1964) 

/ 

Research using^teplication of the present study, in which student-constructed 
test performance is compared with performance on teacher-constructed tests, 
but with the addition of another group in which tlie Ss submit items but 
are then given a teacher constructed examination; and a study of the 
effects of student expectation on actual test performance. (839/1965) 

Study in the areas of further validation of Bloom's taxonomy; item analysis 
data; knowledge as process; triangular bivariate distributions; norms 
for taxonomy-type tests, items, and pyramid tests. (528/1966) 

Research in school settings In which attempts are made to develop and test 
instructions designed to facilitate concept learning in various subject 
fields. (1403/1968) 

Research on construction and validation of reliability criterion measures 
and on environmental influences on success. (1716/1968) 

Research to determine: (1) How one can manipulate the debilitator ' s per- 
formance upward by use of psychometric strategy; (2) whether the effects 
found here are constant across various examination situations and across 
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Investigation of the relationship between test difficulty and study efforts 
and achievement, since the only two studies in this area have equivocal 
findings. (1975/1969) 

Further study of the method of measuring the effectiveness of materials, 
and the extent of structure in materials themselves. (1983/1969) 

Research on the task and cognitive variables which are relevant to per- 
formance, and ways of manipulating these variables. (2236/1969) 

Research into the post-passage review or "backward'^ hypothesis of learning — 
i.e#, that questions inserted into a unit of instruction motivate the 
student to review the preceding material in that unit, (1932/1971) 

Research into the efficacy of the forward or shaping influence hypothesis— 
i.e., that questions inserted into a unit of instruction influence the 
way in which further materials in that unit are studied. (1932/1971) 

Research to explore the effects of knowledge of quiz results on types of 

learning other than memory of factual data, such as application, synthesis, 
analysis, and interpretation of facts and principles. ( /1972) 



C. Test Reliability and Validity 

Research to see if content validity is contingent upon the value judgments 
of those individuals who are defining the term "validity.'' (25A4/1965) 

Study in the areas of further validation of Bloom^s taxonomy; item analysis 
data; knowledge as process; triangular bivariate distributions; norms 
for taxonomy-type tests, items, and pyramid tests. (528/1966) 

Conduct of additional validation studies on the Instructional Objectives 
Preference List. (745/1967) 

Research on construction and validation of reliability criterion measures 
and on environmental influences on success. (1716/1968) 

Research to determine: (1) How one can manipulate the debilltators' per- 
formance upward by use of psychometric strategy; (2) whether the effects 
found here are constant across various examination situations and across 
various kinds of test formats such as completion, matching, recall, etc.; 
and whether it is feasible to unobtrusively program an examination by 
the use of easy and/or difficult items. (1880/1968) 

Research on these assumptions in computer measuring of essay traits: (1) 
Test whether the category definitions used are adequate to the task of 
inducing uniform, reliable grading behavior within and between judges; 
(2) test the validity of the Items chosen to represent the traits; (3) 
determine the extent to which the correlations fluctuate with the essay 
topic; (4) determine if these correlations will be obtained among dif- 
ferent skill levels. (1981/1969) 
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Investigations to compare the discrimination and power of two, three, four, 
and five alternative multiple-choice items. (1992/1970) 

Research on a more reasonable construct validation approach could be based 
on a contrast between: (1) Instructors who had manifested measurable 
skill in pi*omotlng learner attainment of prespeclfled objective, and 
(2) instructors who had not measurably manifested such a skill, (1649/ 
1971) 

Further research to determine whether the present finding can be generalized 
beyond the Likert-type scale to different types of scales (e.g., Osgood's 
Semantic Differential, Thurstone-type scales, graphic rating scales, etc.), 
and other purposes (e.g., the rating of behavior, personality, industrial 
word performance, etc.). It should also be determined whether the con- 
clusions are generalizable to different S populations defined by such 
parameters as level of education or ability, and by psychological, 
experimential , demographic, and ecological characteristics, (1988/1971) 

Research into the effects of leniency in grading essays, especially how it 
varies over time. (2568/1971) 
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SELECT MEDIA/METHODS 



Research needs relating to media have been categorized according to 
media type. Needs related to Instructional methods are further divided 
for each specific method. Also included is a final category of research 
needs specifically related to media comparisons. 



A. Audio Devices 



Research to determine the extent to which discussion periods led by the 
Instructor are necessary for clarification purposes in conjunction with 
tape recorded lecture courses. (2305/1961) 

Studies of: (1) critical listening; (2) the relationship between specific 
listening and reading abilities; (3) ways of exploiting possible re- 
lationships for teaching purposes; (4) teaching techniques and materials 
(5) personality factors which may Influence listening; (6) ways in which 
listening to instruction affects behavior; and (7) measuring devices 
in listening. (1684/1967) 

Studies into thsi relationships between listening research and llnqulstlc 
research. (1684/1967) 

Research in these areas of time-compressed speech techniques: (1) Which 
types of educational material are most suited to time-compressed speech 
presentations? (2) What specific student characteristics Interact most 
closely with the comprehension of time-compressed speech? (817/1968) 

Further research to determine the effectiveness of oral versus visual 
presentation of learning materials, as measured by delayed retention 
of four, eight, and twelve weeks duration. (1600/1968) 

Investigation of the possibility of using time saved by speech compression 
to Improve overall learning. (967/1971) 

Research on Incorporation of visual materials Into a program of speeded- 
speech tapes. (1576/1971) 

Research to determine the maximum length of messages to which Ss can listen 
at given rates without significant losses In comprehension. (2597/1971) 
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B, Visual Devices 



Research oriented toward a systematic evaluation of visual Instructional 
media. This evaluation should be focused on the relative effectiveness 
with which the different types of visuals, possessing varying amounts 
of realistic detail, facilitate student achievement of specific edu- • 
catlonal objectives. (700/1970) 

Research on incorporation of visual materials into a program of speeded- 
speech tapes. (1576/1971) 

Research to explore the relationship on procedure learning of the factors 
of unfamiliar movements and verbal codable movements with moving and 
still sequences. (2195/1971) 

Further I'esearch to identify more precisely whether learning is affected 
by the choice of either printed or projected images. (940/1972) 



C. Audio-Visual Devices 



Research on the pacing of films which require participation and on the 

increase in learning resulting from mental practice of skills demonstrated, 
information conununicated , or questions asked during or after a film showing. 
(2270/1957) 

Research to determine the most appropriate time and the most receptive 
student populations for the use of instructional television-classroom 
settings. (423/1962) 

Research to determine how a visual image affects learning, i.e., by pro- 
viding a sense of actuality which serves as a stimulus, or by implying 
a question which the sound track answers. (2235/1962) 

Research to determine how many statements can be attached to a single image 
or related images before the teaching effect of the image wears out. 
(2235/1962) 

Research to test the retention of materials over varied intervals, when a 
learning film does not reproduce on the screen the material in the 
sound track. (904/1963) 

Evaluation of how many statements can be attached to a single picture or 

related pictures before the teaching effectiveness of the image diminishes. 
(2466/1963) 

Research into these areas: (1) what forms of individual participation of 
students involved in CCTV instruction can be used with large groups of 
students? (2) are there differences in the effectiveness of various 
forms of individual participation? (3) can individual participation 
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activities be developed so that they not only enhance recall but also 
develop problem-solving abilities? (4) what is the effectiveness of 
individual participation in subject areas other than science, and with 
students not on the junior high school level? and (5) what means, in 
addition to forced attention, can be employed to promote greater in- 
volvement with a televised instruction presentation? (841/1965) 

Investigation of these questions: (1) what individual differences affect 
the extent to which Ss are capable of comprehending visual and/or 
auditory modes of presentation? (2) to what extent does familiarity 
influence the ability to comprehend material presented at high rates 
of speed? (3) what are the effects of practice on the comprehension 
of high-speed auditory presentations of meaningful material? (835/1966) 

Additional research to determine the feasibility of telecasting a video- 
taped study-skills! course to university residence halls, libraries, and 
student lounges. (992/1967) 

More research on the effectiveness of the '^Common Learning" television 
course. (2453/1967) 

Investigation of the permanency of the effect produced by a propaganda 
argument delivered by a live or videotaped presentation. (729/1969) 

Investigation of the amount of information retained from a television 
presentation as compared to a live presentation. (729/1969) 

Further investigation of the use of TV in instruction, to possibly include: 
(1) direct and indirect feedback, (2) multiple instruction, and (3) 
TV case studies. (1248/1969). 

More research in instructional television, using intercorrelations fmong 
post-test scores, (696/1971) 

More research in ITV using post-test totals instead of student-totals. 
(696/1971) 

Research replicating this study of the effectiveness of a videotaped model, 
but treating each S as an individual — allowing the S to view the model 
or movie alone. (1949/1971) 

Research to determine if viewing a videotaped model several times can 
enhance the model^s lasting effect. (1949/1971) 

Research to determine what effect there would be on specific behavior if 
a videotaped model was accompanied by comments made by an authority. 
(1949/1971) 

Research on the effective use of video cassettes in both educational and 
industrial applications. (681/1972) 

Media comparison studies using many of the unique contributions of TV 
simultaneously, rather than a single dimension at a time. (698/1972) 
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More detailed investigation of the relationships of visual TV conununi- 
cations techniques to theoretical models of perception and learning. 
(698/1972) 

Testing of color vs. black-and-white effects on learning, opinion, and 
attention with other stimuli, other media, and other methods of 
observing the attention variable (such as recording eye movements) . 
(714/1972) 

Further research through replications of this study (which investigated 
the relative efficacy of advanced organizers, postorganizers , inter- 
spersed questions, and combinations thereof in facilitating learning 
and retention from a televised lecture). One replication should in- 
volve the same experiment at a higher school level* Another should be 
changed only to use simpler organizers. Another should utilize the 
video-scripts as written learning material. (1953/1972) 

Further research on the instructor, content, and student variables in TV 
classes. (2158/1972) 

Further research on audible multi-imagery, to examine; whether it is most 
efficient with certain age groups; what the most efficient career size 
is; what subject matter is best taught by this method; how it can be 
used to help children with reading problems; whether learning is better 
with or without class discussion paired with AMI; hov AMI is related 
to individual differences; what role incidental learning plays in AMI. 
(2313/1972) 

More extensive investigation into the relationship between stimulus 
intensity and vocabulary learning. (2657/1972) 



D. V isual Aids 

Further experimentation on the use of tachlstoscopes and reading rate 
controllers in regard to test performance, eye-movement, age, in- 
telligence, and personality patterns. (2075/1953) 

Research on whether the use of visual aids leads to greater uniformity of 
performance in students. (405/1956) 

Research to determine how a visual image affects learning, i.e. by providing 
a sense of actuality which serves as a stimulus, or by implying a question 
which the sound track answers. (2235/1962) 

Research to determine how many statements can be attached to a single image 
or related images before the teaching effect of the inage wears out. 
(2235/1962) 

Studies to examine the influence of response-dominance levels upon the 
attainment of concepts, when the stimuli are words, pictures, and com- 
binations of words and pictures. (2159/1964) 
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Further research on the use of pictorial embellishment in instructional 
programs. Do the advantages in improved enjoyment for the student 
justify the added cost of such materials? (1552/1965) 

Investigation of ocular movements in relation to display complexity and 
as a function of practice. (1102/1966) 

Research to test the effectiveness of television as a group screening 
method, utilizing a broader sample than the grade-school-age group 
employed in this study. (1427/1967) 

Computerization of the ^'projection tree," or adaptation of it to multiple 
rear-screen setups, or even multiple projection-tree systems. (1891/ 
1967) 

Further research on the following areas: (1) what kinds of subject matter 
are most appropriate for use of the overhead projector? (2) what student 
characteristics would indicate particular receptivity to the use of the 
overhead projector as a supplement to instruction? and (3) to what 
extent does the use of visual aids engender an "illusion of learning" 
among students? (2151/1967) 

Further research on the effects of visual instruction on delayed retention. 
(1461/1969) 

Research oriented toward a systematic evaluation of visual instructional 
media* This evaluation should be focused on the relative effectiveness 
with which the different types of visuals , possessing varying amounts 
of realistic detail, facilitate student achievement of specific 
educational objectives. (700/1970) 

Studies on the effects of pictures on attitudes. More specifically, can 
the use of texts having illustrations that are multiethnic change 
students' attitudes? (1042/1970) 

Research for a graphic image facility, to record, or capture the signal 
Impulses representing graphic information or line-drawing content, 
and to store this information on one of the tape tracks to be programmed 
over one channel of the system, while another track carries the audio 
or verbal Information* (2366/1970) 



E. CAT 



Further research on an information-retrieval system, which would also 
involve research on automatic abstraction, coding, and dissemination 
of information. (939/1963) 

Research on the kinds of information needed by students, teachers, counselors 
and administrators. Also^, development of more standardized methods for 
coding and recording thi^Jinf ormation , so that high-speed processing 
techniques can be used efficiently. (972/1963) 



Further research on the use of the student-response computer system to 
answer questions about the lecture situation • (2344/1966) 

Replication of the present study comparing CAI instruction with traditional 
prograimned text instruction, using a sample population other than the 
Junior high school students used in the present study, would aid in the 
generalizability of present findings, (1922/1967) 

Validation of the credibility of CAI materials* (671/1968) 

Systematic research on the relative effectiveness of analogical and 

digital modes of representation, as these relate to the content of the 
instructional message, to cummunicator and learner characteristics, and 
to the physical and psychological aspects of a particular medium or 
combination of media. (1718/1968) 

Research on the use of computer-managed instruction in post-graduate 
training and review. (2492/1968) 

Application of research to design, and determine validity of, instructional 
techniques and languages which would become available upon acquisition 
of a computer-based instructional system. (660/1970) 

Investigation of available mini-computers to determine the best unit for 
interfacing non-standard terminals and peripherals to various main- 
frame hardware. (660/1970) 

Further research into the use of small computers and computer materials 
in specific application areas. (1287/1970) 

Further research on CAI to investigate the relationship between time taken 
to complete a CAI program and scholastic achievement. (1588/1972) 



F. Programmed Instruction 



Research on what can be used to compensate for the loss of the review-of- 
errors procedure in programmed learning. (2265/1961) 

Further consideration of the multiple-choice type of question in programmed 
instruction. (2561/1961) 

Research in PI, focussed on criterion-specification and measurement (not 
on methods of presentation)* Required are: (1) a generalizable method 
of experimental research which can be applied to the diversity of 
educational products (via learning objectives) ; (2) a common denominator 
upon which comparison between PI and conventional instruction can be 
made (i.e., time or money); and (3) a baseline value that is standardized 
for use across all modes of instruction and will serve as the criterion 
against which instructional results can be measured. (86/1962) 
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Research to answer the following questions; (1) what is the effectiveness 
of programmed Instruction as compared with traditional instruction? 
(2) how adaptable is programmed instruction to changes in course content 
and curriculum? (3) how responsive can programmed instruction be to 
individual student, differences? (4) in what ways does programmed in- 
struction influence student motivation and intent? (5) what is the 
extent of transfer and generalizability of material presented through 
programmed instur ctional methods? and (6) do learning gains stemming 
from programmed instruction justify the additional expense of purchasing 
the necessary machines and programs required to institute this approach 
in a general educational setting? (780/1962) 

Research to test various reinforcement schedules separately on individual 
concepts within programmed instruction^ regardless of the concept's 
order in the total program. (807/1962) 

Further research to determine the effect of intermittent confirmation on 
delayed as well as immediate retention following pror^ianinicd instruction, 
(807/1962) 

Research to test various reinforcement schedules separately on ind.Widual 
concepts within programmed instruction, rega^less of the concept's 
order in the total program. (807/1962) 

Research to determine the exact parameters o. retention over time and the 
corresponding advantages of overt responding. (813/1962) 

Examination of the extent to which studercs actually attend to the context 
of the programmed response. (1339/1 >62) 

Investigation into the amount of ac'uitional time expended by students 

receiving the confirming resprase to programmed material *'in context*" 
(1339/1962) 

Investigation of three ca^cigories of potentially relevant factors for 
programmed instruct"^ ^n : the length of the program, the complexity of 
its content I and '.ne ability level of the learner using a particular 
programming fo'iaat. (1345/1962) 

Research cc'^iaring the relative ef f iciency of over t-versus-c over t response 
modrj in programmed instruction utilizing error rate as an independent 
variable and persistence of wrong responses as a dependent variable. 
(1659/1962) 

Examination of scrambled-versus-ordered sequence with programs of different 
length, in different subject areas, and with students of various age 
levels. (1795/1962) 

Further research to determine the relation of accessibility of information, 
as determined by presentation mode, to mastery of programmed subject 
matter. (1904/1962) 
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Further research to study the interaction between IQ and prograiwned in- 
struction presentation mode, (1904/1962) 

Further research utilizing a programmed logic task to investigate the 
relation between response complexity and implicit responding. (1906/ 
1962) 

Research in the ar^a of programmed instruction, with particular emphasis 
on bypasses and remedial circuits. (1908/1962) 

Further research to determine whether programmed instruction affects 
various ability levels differently, (1910/1962) 

Further research to determine whether differences occur in student per- 
formance resulting from linear-versus-f orward-branching programs, if 
the branching procedures used are highly complex. (1912/1962) 

Examination of the scrambled-versus-ordered sequence question with students 
of various age levels, with programs of different length and in dif- 
ferent subject areas, and with retention tests made at spaced intervals 
of time, (1920/1962) 

Investigations to study Ss performance in a programed sequence on foreign 
language as a function of the mode of item presentation and response. 
Three "modes*' that require experimental comparison are: (1) reading 
the frames only, (2) writing the foreign word when it appears, and 
(3) writing the foreign word when it appears with its pronunciation 
having been presented once, (1390/1963) 

Research to determine whether lower-performing students, who receive 
additional training and consequently improve their scores, retain as 
much as students who scored high originally. (1394/1963) 

Further research on the selective pairing of students in the use of 

programmed materials. What basis for selection yields the most improved 
learning and retention? (1394/1963) 

Research on the following question: Once an adequate sequence of programmed 
materials is developed, can those materials be presented equally 
effectively by machine and text? (1398/1963) 

Further research on the problem of individual differences in performance 
on programmed instruction and the implications of these differences 
to the schools. (1406/1963) 

Instruction of predictors of programmed instruction error rates and learning 
rates. (1406/1963) 

More research into the problem of individual differences in performance on 
programmed instruction, and the implications of these differences for 
the schools, (1406/1963) 
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Further study to answer this question: How do the factors of category of 
response term and location of response word in the preceding sentence 
influence the predictability of the required response and thus in- 
fluence inspection behavior? (1418/1963) 

Further research on what response requirements in programnted Instruction 
lead to complete inspection behaviors. (1418/1963) 

Research on whether goal-attainment time does decrease as a function of 

the degree of group homogeneity. Also, further research on group-versus- 
individual pacing at other age levels and with different disciplines. 
(2261/1963) 

Further research on an investigation of the effects of the pairing of 

students to study programmed materials on the motivation and attitudes 
of the students. (480/1964) 

Research to evaluate systematically the effects of spaced review under a 
variety of instructional conditions. (998/1964) 

Research to determine optimal review spacing procedures in programmed in- 
struction when a number of topics are being taught simultaneously* 
(998/1964) 

Further research on the following question: Given the same instructional 
content, would performance on a corner-clipped looping program and on 
a linear program be equal for Ss without prior knowledge of the course 
content? (1278/1964) 

Study the "practice factor" in relation to revlew-versus-new material, 
and possibly, in relation to the age level of the population for whom 
the program is written. (1436/1964) 

The "discovery" principle in a programmed written instruction manual, and 
comparism to the conventional manual. (1548/1965) 

Research into: (1) the cueing techniques best used for different learners 
and behavioral outcomes, and (2) to resolve the question of "step-isize" 
to be used for different instructional programs. (61/1966) 

Incorporation of both linear and branching formats into a study to determine 
the compatibility of the two types of programs. (730/1966) 

Further research to determine the effect of adjunct programmed instruction 
on transfer of learning. (1282/1966) 

Studies into the influence of programmed instruction upon learning using 
a random sample of teachers and schools for both the experimental and 
control groups. (1316/1966) 

Study of errorless and dialectical programming, with reference to matrix 
algebra, on a retention test. (1562/1966) 
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study of deductive and Inductive programming methods using a delayed 
retention test, (2253/1966) 

Survey of programmed instruction use in a variety of industries. (1804/ 
1967) 

Studies to compare success or failure in programmed instruction with 
individual student characteristics, and to assess which subject area 
would be most appropriate for programmed instruction. (2358/1967) 

Research on the basis for disagreement between time and error measures in 
programmed materials. (2579/1967) 

Continued research to determine the type of program the individual learner 
needs to achieve a particular criterion. (267/1968) 

Study to determine the effects of different feedback conditions upon 
retention over larger periods of time. (973/1968) 

Further research to determine if users of linear programs should be told 
to check their answers only when they are not sure of the answers* 
(973/1968) 

Further experimentation in self-pacing in order that Air Training Command 
''can progress down the road to true self-pacing." (1160/1968) 

Research to control or measure the novelty effect of the programmed method 
when being compared to a conventional method. (1262/1968) 

Research to determine whether the detrimental effect of the rule-of-thumb 
is simply the result of overgeneralization of its use, or whether other 
factors are involved, (1405/1968) 

Further research on the use of programmed learning materials in classrooms, 
to provide remedial work for slow learners and advanced work for fast 
learners, (2210/1968) 

Further studies with a wide variety of programmed materials and with larger 

groups of subjects, to ascertain if the lower correlations of learning 

and verbal intelligence on the part of the students studying low-verbal- 
content programs can be replicated. (728/1969) 

Investigative research on the responsiveness to programmed instruction of 
highly anxious, under-achieving students, (1375/1969) 

Further investigative research on the performance of paired Ss on non-linear 
type programs, (1376/1969) 

Further research to study the effect of variation in block size and step 
size using broader representation of Ss than this particular study 
(eight grades, mean IQ 116). (1463/1969) 
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Research to determine whether It is possible to prepare a program with^ a 
low blackout ratio for content with which S is generally familiar. 
(1592/1969) 

Replacement of attempts to examine the effects of sequence of learning 
measures by attempts to write programs which conform to a defined 
pattern and to develop the appropriate tools for testing these programs, 
(1983/1969) 

Further study on measuring the effectiveness of materials, and the extent 
of structure In materials themselves. (1983/1969) 

Research on the following: Patterns indicative of effective units, 
appropriate variables for the horizontal axis, and mathematical 
descriptions of patterns. Also, identification of relevant basic 
abilities for defined structures, a prerequi$?ite to the meaningful 
study of effectiveness. (1983/1969) 

Implementation of the following in order to fill "gaps" in present 

literature: (1) devote time to empirical analysis of training systems, 
(2) study behavior modification, (3) adopt the PI model for every 
training activity, and (4) measure behavioral outcomes and their 
interactions wlthother systems in an organization and learn the dif- 
ferential effects of competing training strategies. (28/1971) 

Testing of the effect of modifying study habits upon a group receiving 
only programmed instruction. (1936/1971) 



G. Instructional Devices 



Further research on the practicality of utilizing rcading--rate controllers 
and the comparative effectiveness of different approaches to the problem 
of improving reading skills. (937/1954) 

Further validation of specific training devices, where control force is a 
variable. (2273/1954) 

Research to determine whether a reading improvement course in excess of 21 
hours utilizing a reading-rate controller would result in improvement 
in reading speed equal to or greater than that evidenced in a book- 
centered course. (2625/1954) 

Additional research into the complexities of language learning and further 
exploitation of the language laboratory by interdisciplinary teams. 
(2671/1962) 

Research on training teachers to use an automated classroom. (942/1963) 

Further comparative research on the "adjunct auto'-lnstruction" program, 
and programmed learning methods. (2007/1963) 
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Further research to test the effectiveness of a self-scoring instructional 
device in an introductory psychology course, using a sample larger than 
52. (1315/1965) 

Utilization of the language laboratory in recombinlng linguistic concepts 
and recording short stories and plays, (2591/1968) 

Continued research to improve educational technology using efforts which 
permit effective interaction of hardware and software development, 
(2184/1969) 

Investigation of the overall relationship of the factors which go into, 
instruction and administration of a course: (1) what kind and how 
many of the Army's training requirements are amenable to automated 
instructional techniques? (2) what kind of instructors or supervisors 
would be required for partially or fully automated courses? (3) can 
purely automated courses maintain active student interest over long 
periods of time? and (4) how can automated courses which emit students 
individually rather than in groups be meshed together into an integrated 
training program? (722/1970) 

Further research on an expanded computer-controlled teaching machine that 
will give automated instruction to 20 students simultaneously. (2607/ 
1971) 

Research in the following broad areas: the role of modern technology in 
shaping contemporary Soviet institutions; the human-electronic symbiotic 
relationships in the Soviet classroom; the role of social engineers in 
Soviet classrooms; and the Soviet efforts to refine and disseminate the 
hardware and software needed to automate instruction. (1565/1972) 



H, Methods of Instruction 



Lecture Discussion 



Sharpening of theories and methodology in further research focused on 
student-centered versus instructor-centered teaching. (2120/1954) 

Further research in the general area of improving the teaching of written 
composition. (1680/1967) 

Investigation into the scheduling of note-taking via cues, which could be 
manipulated by structuring the lecture material. (1757/1968) 

Investigation of the relationship between learner achievement and principles 
of instruction in an experimental design rather than in a correlational 
context. (1279/1969) 
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Research to answer these questions: (1) in what ways do (a) length of 
lecture I (b) method of testing, (c) manner of instruction, and/or 
(d) isolation effects influence the serial position curve for immediate 
recall of lecture material? and, (2) do differences in the serial 
position effect occur as a function of tape recorded versus live 
lecture presentations? (1333/1969) 

Research on a flexible lecture-discussion schedule and its effects on 
cognitive and affective learning and teacher-student relationships, 
(971/1970) 

Further research investigating student participation in role playing, and 
comparing the traditional lecture method to taped lectures, movies, 
television, reading, dialogue, and group interaction. (2480/1970) 

Identification and location of sources of supplemental materials for use 
in Independent study. (1580/1972) 

Research to determine how much and what kind of reinforcement should be 
used when employing behavioral objectives with independent learning. 
(1580/1972) 

Studies to replicate this one on the use of behavioral objectives, and 
independent study with different subject samples. (1580/1972) 

Further research to determine whether dependent Ss perform better under 
a traditional lecture environment t^an under an independent study 
plan. (1584/1972) 

Further research on the use of adjunct questions in lecture situations. 



(2421/1972) 




Team Teaching . • 

Research to determine the extent to which various aspects of the team 
system in use at the College of Basic Studies of Boston University 
account for the unusual degree of scholastic growth shown by students 
in that program. (789/1964) 

Investigation of the relationship between a student's academic performance 
in a team-teaching situation and his field of concentration. (785/ 
1967) 

Investigation into the following questions: (1) what type of course 
material is most conducive to team-teaching instruction? (2) are 
high-ability students more receptive to a team-teaching approach than 
low-ability students; and (3) do team-teaching approaches place dif- 
ferential emphasis on modes of learning as compared with traditional 
classroom approaches? (1377/1967) 
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simulation 



Research to ascertain whether there Is a significantly greater value In a 
ground-based trainer cockpit that Is an exact replica of the aircraft 
used In training, (1564/1966) 

Determination of the results of using a syllabus that Introduces and 

practices each maneuver and procedure In the trainer before It Is intro- 
duced In the aircraft. (1564/1966) 

Development of evaluation procedures for simulation games. (577/1970) 

Studies on: (1) development of the form and content of task Information 
so that output Is directly useful In decisions on training device 
design, (2) formal approaches for determining training value received 
for equipment costs expended, (3) amount of display degradation 
tolerance without performance degradation, and (4) transfer of training 
benefits that accrue from device options that deliberately depart from 
realism. (519/1971) 

Research to compare two or more simulation games v/hich have been developed 
to teach the same facta, abilities, and skills. (575/1971) 

Research to deteirmine the advantages or disadvantages of using simulation 
games with disadvantaged and/or non-verbal subjects. (575/1971) 



Tutorial Instruction 



* . 

Research to;investigate the characteristics of those students who fail and 

# those who succeed in a tutorial program. Also, study of the variability 
^ which exists within a tutorial program itself. (1924/1967) 

^dsearch to determine the role motivation plays in academic performance. 
Wh^t results would be found if the tutored were matched with a tutor- 
motl<rated sample which was denied tutoring? (1892/1969) 

Research into the effects of a remedial tutorial program using a large 
sample, and making subject-by-subject comparisons. (1892/1969) 



I. Media Comparisons 



Research to compare tape-recorded instruction versus more conventional 
correspondence courses. (2237/1962) 

Further research to: (1) test the feasibility of the guidelines proposed 
in th^s 8fudy, (2) Identify heuristic principles available to evaluate 
the role of the media specialist, (3) build a research-hypotheses 
generating mechanism into the course development procedures to identify 
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unknowns particular to this process and make possible more integrated 
and applicable research, (4) explore the nature of training programs 
for specialists in instructional design and media applications to 
teaching, and (5) disseminate the findings of successful media 
innovations and instructional developments, to broaden the base of 
these efforts and deliver their benefits to more universities and 
colleges. (529/1965) 

Improvement in future media research to obtain either situation-specific 
findings or generalizable principles. Also, creation of a "media 
taxonomy" listing commonly known and unusual features, stimulus 
dimensions, and functions to be served by the various media. (16/1966) 

Replication of this study utilizing additional treatment conditions of 

program-program, text-text, and lecture-lecture in order to investigate 
more thoroughly the effects of repetition on learning, (730/1966) 

Research to answer these questions: How does one medium interact with 
other media, different types of learners, different types of teachers, 
and different class sizes? To what extent can media assume the in- 
structional function? What administrative patterns and arrangements 
are appropriate in schools committed to a technology of instruction? 
(1674/1966) 

Research to determine the effect of dial-access retrieval on: (1) student 
learning (grades and performance), (2) study patterns, (3) role of 
teacher and use of time, (4) curriculum, (5) methods, (6) role of 
library, and (7) cost-effectiveness. (661/1968) 

Research to investigate the potential of developments in compressed speech, 
single-concept films, and programmed media, and theii effects on dial- 
access systems, (661/1968) 

Research on the integrated use of dial access with other modern instruc- 
tional technologies, such as instructional TV, radio, computers, 
electronic response systems, etc. (661/1968) 

Further data on cost-effectiveness of various media. (1197/1968) 

Research on more effective utilization of various combinations of media. 
(1197/1968) 

Analysis of motion picture and TV research in terms of current production 
and utilization data. (1722/1968) 

Research into the relationships between unique media attributes and their 
physiological functions under specific task requirements and specific 
learners. Also, continued research on four effects of mass communication: 
(1) aptitudinal, (2) attitudinal, (3) emotional, and (4) social. (247/ 
1970) 
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Investigation into the following: (1) traditional nature of instructional 
content, (2) effectiveness of various media, (3) teacher training, (4) 
student attitudes (motivation, self-reinforcement), (5) determination 
of learning experiences of value, and (6) efficiency and cost- 
effectiveness of instruction, (59/1971) 

Further investigation of the interaction between aptitude and type of 
modeling treatment (written versus video) as teacher training using 
refined aptitude and treatment variables and data such as consideration 
of cost factors. (2006/1971) 

Media comparison studies which use different forms of dependent measures to 
measure learning. (698/1972) 

Media comparison studies using many of the unique contributions of TV 
simultaneously, rather than a single dimension at a time. (698/1972) 

Investigation into the relationships of visual TV communications techniques 
to theoretical models of perception and learning, (698/1972) 

Testing of color versus black-and-white effects on learning, opinion, and 
attention with other stimuli, other media, and other methods of observing 
the attention variable (such as recording eye movements), (71A/1972) 
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DEVELOP INSTRUCTIONAL MATERIALS 



The research needs related to this process have been divided Into 
three categories. The category "Learning Theories^' includes discussions 
and suggestions for research on psychological learning theories, such as 
behavior modification and behavior acquisition models. The "instructional 
Techniques" category includes needs for research on topics such as the 
value of lecture versus discussion and recitation versus written tests. 
Included in the last category, "Factors Affecting Learning," are research 
suggestions ranging from the effects of test anxiety to the problem of 
retroactive inhibition. 



A* Learning Theories 



Study of the problem-solving processes involved in concept learning. 
(2121/1957) 

Research to determine whether observed differences in concept learning 

between searching and dependent groups were due to motivational charac- 
teristics. (2121/1957) 

Research into possible differential expectancies toward the experimenter's 
validational responses in relation to concept learning, (1918/1962) 

Further research in language learning, following the pattern of analysis 
of verbal behavior suggested by B. F. Skinner. (1390/1963) 

Development of mf»aningful tetmiaology and labels (in addition to the 
important but general tem "reinforcing stimuli") to d<^scribe and 
classify the stimulus variables employed in research on discovery 
learning* A sophisticated theory of concept formation Is needed, not 
only to supply these terns and labels, but also to account for phenomena 
observed in research on learning by discovery. (1824/1963) 

Research to determine whether specific sorts of feedback are needed to 
make specific kinds of decisions. (1439/1967) 

Research to operationally define, experimentally, terms such as "cybernetic 
control," "criterion stimuli," and ^'negative feedback," and demonstrate 
how these events interact in an educational context. (1692/1968) 

Further research on problem-solving strategies, rather than problem-solving 
choice responses* (1318/1969) 
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Research into the relationships between unique media attributes and their 
physiological functions, under specific task requirements and for 
specific learners, Also, continuation of research on four effects of 
mass communication: (1) aptitudinal, (2) attitudinaJ. , (3) emotional, 
and (4) social. (247/1970) 

Research to determine what kind of learning activities can be made con- 
tingent upon the discriminative text cues that are defined by questions. 
(1048/1970) 

Research directed "toward the studies of stimulus encoding processes, 
retrieval from short-term memory and long-term memory, stimulus 
redundancy and between-channel redundancy, and the disinhibition 
process of proactive and retroactive inhibitions." (693/1971) 

Research on schedules of reinforcement, the immediacy of the incentive 
provided, the curriculum step-size, and the reward contingent on the 
successful performance of each step. (1256/1971) 

Research into the post-passage review or "backward" hypothesis of learning — 
i.e., that questions inserted into a unit of instruction motivate the 
student to review the preceding material in that unit. (1932/1971) 

Research on the efficacy of the forward or shaping influence hypothesis — 
i.e., that questions inserted into a unit of instruction influence the 
way in which further materials in that unit are studied. (1932/1971) 

Research to detemine the type of concepts that can be learned most 
efficiently through the use of behavioral objectives. (1580/1972) 



B. Instructional Techniques 



Research to examine specific characteristics of various teaching methods 
as independent variables, in addition to the overall method taken as a 
single independent variable. (1319/1963) 

Study of the reliability of the stimulated response, technique via a test- 
retest situation. (1803/1963) 

Investigation of differences in the relevant thought stimulated by different 
subject areas. (1803/1963) 

Additional study of constant delay of reinforcement. There is no study 
of the relationship between a range of possible competing response (as 
one type of interference) and resistance to extinction. Also, research 
is needed into the effectiveness of delayed punishment versus delayed 
reward which has also not been explored. (424/1964) 
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Investigation of how personality variables influence the efficiency of 
teaching methods. (1725/1964) 

Investigation of the hypothesis that the method of instruction has dif- 
ferent effects on different ability groups, (1725/1964) 

Further testing of the hypothesis that anxiety seems to have no effect 
on learning* (1725/1964) 

Investigation to ascertain the relative merit of explanatory versus 

non-explanatory reinforcement, as measured by performance on objective 
examinations. (1763/1965) 

Research into: (1) the cueing techniques best used for different learners 
and behavioral outcomes, and (2) to resolve the question of "step-size*' 
to be used for different instructional programs. (61/1966) 

Research to determine how best to encourage teachers to utilize curriculum 
guides. Is having them participate in the construction of the guides 
helpful? (1676/1966) 

More study of complex motor activity, involving not only the execution of 
the individual motor components, but also their periodic assessment as 
the S moves through the entire sequence of the task. (1670/1967) 

Studies to determine which aspects of a complex motor skill can be learned 
through mental practice, and which cannot. (1670/1967) 

Research on the use of a different cueing procedure, such as sound intensity, 
light intensity, both, or a combination of cues to maintain student 
attention. (1757/1968) 

Investigation into the scheduling of note-taking via cues, which could be 
manipulated by structuring the lecture material. (1757/1968) 

Research to answer these questions: Why is the recitation method so 
successful in the evolutionary struggle with other, more highly 
recommended methods? What survival needs of teachers are uniquely 
met by the recitation? Why have teacher training institutions been 
unable to defer teachers from using recitation? Is recitation the 
best or most practicable pedagogical method? (1072/1969) 

Research to test Ausubel's hypothesis that discovery learning is more 

relevant to children, and reception learning is more relevant to older 
students. (1640/1969) 

Research on transfer of training over periods in excess of six weeks, in 
discovery learning. (1640/1969) 

Research to determine the relevance of Gagn^s categories of learning to 
the discovery method. (1640/1969) j 
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Research tc examine savings on relearning In discovery learning. (1640/ 
1969) 



Study of feedback methods, with the addition of a delayed retention 
measure. (2142/1969) 

Study to Contrast the use of different cloze techniques in reading in- 
structions. (2439/1969) 

Further research on a word-association technique for a military radar 
maintenance course. (554/1970) 

Extension of the consciousness-sampling method across a variety of topics 
and themes. (791/1970) 

Development of a taxonomy of teaching behaviors, and simulated classroom 
situations focusing on these teaching behaviors. (1285/1970) 

\ 

Investigation of the possibility that subject-produced responses may 
cause proactive or retroactive interference. (1302/1970) 

Research on identification and shaping of specific methegenic or attention- 
like behaviors through the use of properly structured material. 
(1985/1970) 

Research to attach practical significance to the results of instructional- 
technique/individual-differences types of studies. (854/1971) 



C * Facto rs Affect:|.ng Learning 

Research to determine if verbal description without performance and/or 
verbalization over more than one trial would exert a positive effect 
on the ability of a learner to perforin a task. (2223/1952) 

Investigation of the effect of length of summary and introductory sequences 
on learning. (755/1953) 

Further research to substantiate the hypothesis that retroactive inhibition 
makes an important contribution to the? forgetting of meaningful material. 
(995/1955) 

Use of a population different from that in the present study, to examine 
the effect of organizational aids on comprehension and retention* 
(2077/1955) 

Further research in reading Improvement, considering variables such as 
personality structure-learning interaction, perceptual development, 
and treatment variables (especially the instructor). (2079/1955) 
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Research to determine the effect of temperature upon technical training 
in industrial and other civilian situations. (2179/1955) 

Further research on the relationship of style difficulty to retention, 
especially where "compensation*' may be a factor. (2199/1955) 

Research into possible differential expectancies toward the experimenter's 
validational responses in relation to concept learning, (1918/1962) 

Additional research to examine the scrambled-versus-ordered sequence 
question with students of various age levels, with programs of 
different length and in different subject areas, and with retention 
tests made at spaced intervals of time. (1920/1962) 

Systematic research into response-mode effects with controls for program 
length and length of retention interval, (1368/1963) 

Research to determine whether or not students with higher entering 

behaviors than that of a course's target population can be Instructed 
to make covert responses, (1378/1963) 

Investigations to determine the most efficient use, with regards to time, 
of overt and covert responding. When is it best to require the overt 
response, and when is the information level high enough? (1378/1963) 

Research utilizing an informational approach in studying incidental 
learning. (1378/1963) 

Additional study of constant delay of reinforcement. There is no study 
of the relationship between a range of possible competing response 
(as one type of interference) and resistance to extinction. The 
effectiveness of delayed punishment versus delayed reward has not 
been explored, and is recommended as an area for research* (424/1964) 

Additional research to measure the authoritarianism/achievement relation 
in the learning of students in industrial training classes, (479/1964) 

Research to investigate the degree to which the adroit timing of exposure 
in different sense modalities can facilitate learning, as a function of 
different linguistic problems and differences in individual learning 
ability, (1791/1964) 

Further investigation to study the effects that long-term training with 
ambiguous stimuli has on discrimination of unambiguous stimuli. 
(1350/1965) 

Further investigation to determine the effect that controlled variation 
of figure ambiguity throughout training has on discrimination of 
figures, (1350/1965) 



Investigations of modifiable monitoring behavior, to precede future 
research on perceptual training methods, (1102/1966) 



Further investigative studies on complex perceptual tasks. (1102/1966) 



Empirical examination of the concept of data display complexity in a target 
detection task. (1102/1966) 

Research to determine which condition corresponds with the greater job 
output — knowledge of results given after doing the task for some time, 
or knowledge of results given from the time the task is first begun. 
(1290/1966) 

Investigations of the effect that knowledge of performance has on job 
output, using larger samples to reduce inter-indivldual variation, 
(1290/1966) 

Continuation of research on the exact mechanism by which learning behaviors 
are modified by questions appearing Immediately after the text* (779/ 
1967) 

Research to detemine If different limits of learning using accelerated 
speech result from distributing learning over a greater span of time 
(such as 30 minutes). (1957/1967) 

Determination of the effects of presentation order of advanced organizers. 
The question of how learning efficiency depends on the order of 
presenting abstractions and concrete models of these abstractions is 
important to curriculum reformers in mathematics and should be in- 
vestigated experimentally. (1963/1967) 

Additional research in these areas: (1) what specific elements of the 
mnemonic system are responsible for facilitating recall? (2) what 
external variables influence the performance of Ss using a mnemonic 
system? (3) would the use of imagery £r mediation facilitate the 
recall of Ss as compared with a control group given standard paired- 
associate instructions? (4) does presentation of a peg-word list 
facilitate learning a word list which requires a considerable amount 
of response learning? (5) what is the rate of recall for Ss learning 
a single list with different presentation rates under mediation or 
nonmedlation instructions? (6) how effective is a mnemonic system in 
learning more complex material? and (7) what Is the long-term 
retention of material learned by means of a peg-list approach? (2430/ 
1967) 

Research to determine if cues such as delayed Information have any effect 
on Immediate and delayed retention, (803/1968) 

Study of the effects of different feedback conditions upon retention, over 
larger periods of time, (973/1968) 

Research Into what forms psychometric feedback should take, so as to 
produce the best effects, (1395/1968) 



Investigation of the hypothesis that when task-appropriate behaviors do 
not exist in S's repertoire, postquestions would selectively reinforce 
the more appropriate behaviors* This instrumental learning should be 
reflected in a gradual improvement of the posttreatment over the pre- 
treatment groups over time, (1399/1968) 

Research to determine whether the detrimental effect of the rule-of -thumb 
is simply the result of overgeneralization of its use, or whether other 
factors are involved. (1405/1968) 

Further experimentation to test the effect of specific variations in 

treatment. For example, under what conditions is informational feed- 
back of great help and under what conditions does it interfere with 
learning? What does a graduated sequence of information take if feed- 
back is to be of greatest help? How does increasing the length of the 
scenes to include a greater number of observed behaviors influence the 
observer's performance? (1971/1968) 

Further investigation of the relevant behaviors which underlie or 

strongly interface with skillful observation of intended pupil per-- 
formance may be profitable. (1971/1968) 

Research to determine the effects of the rank and general attitude of the 
class leader on performance in a military training class. (2418/1968) 

Research to determine the effect of competition groups (formed on the 
basis of peer ratings) upon student performance. (2418/1968) 

Research to determine the effect on performance in a military training 
class of making exams more or less difficult during the first weeks 
of training. (2418/1968) 

Further research on instruments to measure learner characteristics other 
than achievement, and development of a variety of materials designed 
to meet the individual learning styles of the student. (2489/1968) 

Studies of goals and intentions as a function of the incentives provided 
by E. (1080/1969) 

Studies of the form of Incentives given to Ss as they relate to motivation, 
(1080/1969) 

Experimental study of the differential effects of familiarity of "things," 
"actions," and "relations" recall of meaningful propositions. (1310/ 
1969) 

Research to determine whether the post-TLE decline on latency is actually 
indicative of the growth of associative strength, (1338/1969) 

Research to determine if the degree of retention can be controlled by 
training to a latency criterion. (1338/1969) 



Research to determine latency changes over the course of learning in more 
complex tasks, such as concept foirmation, (1338/1969) 

Research to determine if latency measures have utility for instructional 
decisions in the early stages of the task. (1338/1969) 

Research to determine if response latencies reach a stable asymptote, and 
if so, whether this asymptote has any implications for retention. 
(1338/1969) 

Research to answer the question, ''Does low homogeneous ability pairing 
provide an optimal condition for learning conceptual materials during 
a brief exposure?'^ (1392/1969) 

Further study to investigate the hypothesis that with delay of Information 
Feedback following a S's response, the S responds to the presentation 
of additional cues or stimulus aspects of IF, thus learning more about 
the item; and that when these cues can be used in retention, delay 
Improves retention. (1412/1969) 

Research into the effect of variables, such as semantic or syntactic 

transformations and irrelevant information, on productive and reproductive 
memory . (1016/1970) 

Research to determine what kind of learning activities can be made con- 
tingent upon the discriminative text cues that are defined by questions. 
(1048/1970) 

Research on these questions: What are the more critical student activities 
that affect what is learned in the more Important institutional settings? 
What are the factors that modify mathemagenic activities? What causes 
positive mathemagenic activities to deteriorate? How can one give a 
coherent account of Class III activities that is consistent with what 
is known about visual perception, memory, and language processes? 
(1050/1970) 

Research on delineating and/or relating considerations of some of the 
following variables which may be associated with the learning set of. 
individuals: instructional content, Instructional strategy, various 
learner characteristics, and/or levels of student cognitive functioning. 
(1186/1970) 

Research to determine the effects of formal training on strategies used in 
concept attainment. (2610/1970) 

Research investigating the effects on learning of pairing students when 
students choose their own partners* (699/1971) 

Further research to investigate the effect of feedback stimuli on the 
learning of the teacher who is instructing a responding learner. 
(763/1971) 



Further research to examine various teaching-learning processes to 

determine additional benefits of a teaching experience on the learning 
of the teacher, and to analyze those bpnefits into the appropriate 
sources of transfer. Possible benefits are strategies of responding, 
rate of responding, and types of errors and correct responses. (763/ 
1971) 

Further research to demonstrate the generalizability of the effects of 

using a warning as an instructional strategy, by using different subject 
matters or problem solving situations. (823/1971) 

Research to determine if vicarious reinforcement can be used to increase 
performance in academic settings. (1064/1971) 

Investigation of the effects of labeling stimuli and S-R contingencies in 
training and extradimensional-constant transfer after different amounts 
of training. In this context, It would be useful to investigate the 
relationship of labeling, discrimination of change between training 
and transfer, and the commission of perserverative errors to the 
previously relevant training cue and dimension. (1608/1971) 

Research to determine whether the learning consequences of an instructional 
aid react differentially upon various components of the instructional 
episode • Does the use of an inferential adjunct question tend to cause 
recall of so much toxt material that it interferes with the recall of 
the question itself? Does such interference diminish if the text is 
learned more completely? (1995/1971) 

Research to determine if it is possible to provide irrelevant information 
to Ss to cause them to search for new and better ways of discovering 
rules—a positive effect of irrelevant information. (1998/1971) 

Study to determine whether having Ss skim through material they have read 
once Improves their mastery of the subject matter to the same level as 
working an instructional program* (2003/1971) 

Investigations to determine if familiarity with subject matter could be a 
useful predictor of optimal instructional strategies. (2003/1971) 

Investigations into the possible relationships that may exist between type 
of prior reinf orceinent given and other organismic variables. (2010/ 
1971) 

Research to determine whether one can inhibit or facilitate S performance 
in academic and non-academic settings, by variation of the nature of 
the Information given the S concerning his potential for performance in 
a given area* (2010/1971) 

Further study on the effects of informal verbal feedback on self- 
perception during sensitivity training. (2144/1971) 
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Studies on the relationship between teaching methods and the area of 

principles learning, using teachers train«id in the particular methods 
to be used. (2147/1971) 

Research to explore the relationship on procedure learning of the factors 
of unfamiliar movements and verbal codable movements with moving and 
still sequences. (2195/1971) 

Investigation of the relationships between the three variables- -augmentive 
Information, instance difficulty, and method of presentation — in terms 
of the amount and type of information being presented by the instances, 
what types of learner strategies are employed, and the degree of task 
complexity. (896/1972) 

Research in educational and applied learning situations to determine 
whether the principle that intratask Interference during learning 
facilitates retention and transter can be generalized to nonlaboratory 
situations. (956/1972) 

More research to provide a sound foundation in the experimental analysis 
of human academic learning. (976/1972) 

Future research to explore ways to maximize the effects on learning of 
compl(2X combinations of variables, including objectives. (2165/1972) 

Evaluation of the Internal-External Control task setting with a variety 
of social and ethnic groups. (2510/1972) 

Research to determine the effect of different question construction (type, 
content, or form of question) on learning of textual material. (2393/ 
1973) 



EVALUATE INSTRUCTIONAL PROGRAM 



Included in this category of research needs are suggestions for 
evaluation of the instructional program. Research needs were further 
divided into determining the effects of instructor, student and environ- 
mental factors because it was determined that outside factors, rather 
than the actual content of the program, may hinder or facilitate program 
effectiveness. 



A. Analyze/Evaluate Program Data 



Definition of the range of variables and measures to serve as the bases 
for program evaluation. (1728/1968) 

Study on adapting the methods of the social and behavioral sciences to 
educational evaluation. (1052/1970) 

Prototypical studies Using matrix sampling and generalizability theory in 
educational evaluation. (1052/1970) 

Regular evaluation of the relevance of technical curricula to the educationa 
Input into the labor market. (1104/1970) 

Research to develop more refined forecasting techniques, especially those 
used to identify the impact of technological changes on skill require- 
ments and demand for labor. (1104/1970) 



B. Evaluate Instructors 



Instructor Effectiveness ^ 

Investigation of the following questions: (1) what is the comparative 
effectiveness of certified versus uncertified trainers? (2) does a 
warm and friendly trainer produce a more self-initiated and work-oriented 
student? (3) what is the effect of social competence on overall training 
competence? and (4) what trainer characteristics are related to the 
competence of a trainer, as reflected by measurable changes in trainee 
behavior? (1383/1967) 

Research to determine why englneierlng students prefer Instructors who 
lecture, and who conduct factually oriented, controlled classes* Is 
it because this style is particularly suited to the course material, or 
because the Instructors consider it to be the njost appropriate style, or 
because the style is supportive of the personal needs of the 'instructors? 
(1018/1970) 



Evaluation of Instructor rating systems to determine If they are truly 
related to important educational criteria, such as student achievement 
In the school and student performance on the job. (711/1970) 

Further testing to Investigate the Idea of teacher evaluation as an 
estimation of discrepancies between Ideal and observed behaviors. 
(2014/1971) 

Investigation of the questions "for which objectives?'' and "for which 
students?" In order to answer the question "which teachers are most 
effective?" 



Personality Variables^ 

More research to determine the interaction between student personality and 
specific teacher-student interview techniques. (989/1956) 

More research to determine the types of teachers who could succeed in 
teacher-student interviewing. (989/1956) 

Research to answer the following questions: (1) what is the efficacy of 
various teaching methods independent of the teacher's individual 
personality? (2) in what ways do different teacher personalities 
enhance or detract from various instructional methods? (776/1962) 

Research to answer the following questions: (1) what is the relationship 
between the actual progress of a student as measured by standardized 
tests, and the instructors ratings? (2) in what ways are instructors' 
perceptions of students affected by personality variables? and (3) 
is the way that an instructor grades a student related to that 
instructor's own personality characteristics? (2355/1965) 

Investigation of these questions: Can teacher trainees whose conceptual 
systems are at lower stages of development be taught to use a reflective 
teaching style in classroom teaching performance? What conditions are 
sufficient to effect such a change? (1074/1970) 

Research to determine the extent to which the study results (student 
behavior influences teacher behavior in predictable directions) 
generalize to other non-education courses, with regular (not guest) 
teachers. Consider research in which (1) input variables such as student 
and teacher characteristics which the individual possesses before entering 
the class, (2) verbal and non-verbal cognitive and affective student and 
teacher classroom interactions, and (3) student and teacher products 
measured after a designated interval. (2030/1971) 
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Training of Instructors 



Increased research in the use of nonprofessionals in selection and training 
procedures, relationship with teachers and students, aids effectiveness 
at various tasks, and resultant changes in teacher work patterns. 
(1010/1967) 

Study of preservice-inservice relationships, teacher role differentiation, 
programs for teachers of culturally disadvantaged youth, and programs 
built around the integrated use of newer media* (1012/1967) 

Research to contrast teachers who have been thoroughly trained in the 
use of behavioral objectives with an untrained group of teachers on 
the basis of their students' achievement. (1767/1969) 

Re.search into the application of suitable contingency models for rein- 
forcing measured changes in student behavior* (1767/1969) 

Study of the relationship between teaching methods and the area of 

principles learning, using teachers trained in the particular methods 
to be used. (2147/1971) 

Continuation of research on the educational value of the individualized 

competency-based system of teacher education described in this document, 
(2586/1972) 



C . Investigate Student Characteristics 

Research to detetmine if students whose low verbal-aptitude scores are 

apparently due to poor reading ability, can Improve their verbal-aptitud 
scores after a period of intensive instruction in reading. (1916/1963) 

Further study to determine if persons who make great gains In a reading 
Improvement course are those persons whose initial reading scores are • 
low and who are emotionally stable and non-anxious, (1916/1963) 

Research to determine which types of students are apt to benefit most from 
an independent study approach to education. (833/1966) 

Research in these areas: (1) what kinds of subject matter are most 

appropriate for use of the overhead projector? (2) what student charac- 
teristics would indicate particular receptivity to the use of the over- 
head projector as a supplement to instruct? and (3) to what extent does 
the use of visual aids engender an ''illustion of learning" among 
students? (2151/1967) 

Further research to directly test the hypothesis that decline in Inspection 
time is directly related to a lessening of mathemagenic (attention-like 
responses) behavior resulting in diminished learning effects. {1273/ 



Replication of the entire series of studies on classroom climate and 
individual learning, with revised and more reliable instru^ients , and 
using a national random sample. (1409/1968) 

Research to identify the variables along which student attitude toward 
instructional media are differentiated. (1570/1968) 

Research on the effects of the environment, traced on individual learners 
grouped by mental ability, biographical characteristics, and personality* 
(1306/1969) 

Research to identify experimentally manipulated factors and status variables 
which affect the social environment. (1306/1969) 

Comprehensive and systematic research on affective variables in academic 
achievement. (1352/1969) 

Development of instruments which differentiate between the self-description 
and self-evaluation aspects of self -concept . These should then be 
related to various criteria, to ascertain the relative effectiveness 
of the two domains. (1035/1970) 

Development of instruments to assess self-perceptions of past and future 
and to determine their strength in predicting academic success. (1035/ 
1970) 

Research in the area of self-perception as a predictor of academic success 
for women. Why do women's perceptions of themselves not relate to their 
performance? (1035/1970) 

Research on the following questions: What are the more critical student 

activities that affect what is learned in the more important institutional 
settings? What are the factors that modify mathemagenic activities? What 
causes positive mathemagenic activities to deteriorate? How can one give 
a coherent account of Class III activities that is consistent with what 
is known about visual perception, memory, and language processes? (1050/ 
1970) 

More research to develop techniques for developing^positive attitudes and 
modifying negative attitudes toward mathematics. (1660/1970) 

Longitudinal studies on attitudes toward mathematics, and studies utilizing 
multivariate designs which combine the correlational and experimental 
approaches. (1660/1970) 

Further research on the effects of class characteristics on learning, 
(1990/1970) 

Research to explore sex-differentiated effects of the peer group in school 
classes. (1990/1970) 



Further research focused on class subgroups to determine the effects of 
subgroup characteristics on learning. (1990/1970) 



Research to determine whether students participating In a performance 

contract-Individualized instruction type of program have poorer achieve- 
ment when allowed to select their own contracts on the basis of preference 
than when contracts are presented In a formal sequence. (2385/1970) 

Further research attaching practical significance to the results of 

instructional-technlque/lndlvldual-dlf ference type of studies, (854/ 
1971) 

Research to measure the Interaction of student opinion and the grade level 
for which the student contracted In grade-contract situations, (862/ 
1971) 

Research into individual differences in processing listening materials. 
The method of speech compression could be utilized to equate the per- 
formance of high- and low-aptitude Ss, thus allowing for the study of 
differences in the type of information transmitted. (961/1971) 

Additional research to discover what variables (besides achievement 

orientation) are of interactive significance to academic achievement. 
(1926/1971) 

Research to determine the extent to which the study results (student 
behavior influences teacher behavior in predictable directions) 
generalize to other non-education courses with regular, not guest 
teachers. Consider research in which input variables, such as the 
following are considered: (1) student and teacher characteristics 
which the individual possess before entering the class, (2) verbal 
and non-verbal cognitive and effective student and teacher classroom 
interactions, and (3) student and teacher products measured after a 
designated interval. (2030/1971) 

Systematic exploration of the relationship of the interest and attitude 
of the learner toward subsequent learning, with appropriate dependent 
variables. (698/1972) 

Additional research to assess the ways in which individual-differences 
data, as related to some given material to be learned, may be used in 
a manner most accoirmodating to the needs of the Individual student. 
(756/1972) 



D . Investigate Effects of Student-Instructor Interactions 



More research to determine the types of teachers who could succeed in 
teacher-student interviewing. (989/1956) 



More research to determine the Interaction between student personality and 
specific teacher-student interview techniques. (989/1956) 



More research to determine the effects of teacher-student interviews on 
the perfoirmance of borderline achievers* (989/1956) 



Research on the extent to which the amount or kind of participation by 
the more social student may vary with the instructor's presence or 
absence in group discussions. (1673/1960) 

Research to answer the following questions: (1) what is the relationship 
between the actual progress of a student as measured by standardized 
tests, and the instructors ratings? (2) in what ways are instructors' 
perceptions of students affected by personality variables? and (3) 
is the way that an instructor grades a student related to that in- 
structor's own personality characteristics? (2355/1965) 

Assessment of the individual aviation instructor's total influence on 
student achievement. (1564/1966) 

Increased research in the use of nonprofessionals in selection and training 
procedures, relationship with teachers and students, aids effectiveness 
at various tasks, and resultant changes in teacher work patterns. 
(1010/1967) 

Research devoted to discovering aspects of the individualized systems 
approach that would alleviate student feelings of isolation, (793/ 
1970) 

Further investigation to relate teacher behavior with resulting student 
performance. This should use an observational system (rating or 
category) and measures of class achievement adjusted for initial 
aptitude or ability. (1054/1970) 

Further research on the effects of teacher attitudes on student attitudes. 
Methods used in this area should involve direct observation of teacher- 
pupil interaction and measures of teacher expectations in mathematics. 
(1660/1970) 

Continuation of research in the area of student-teacher personal interaction, 
including larger sample sizes and using formal records of both student- 
and teacher-initiated student-teacher personal interations. (1711/1971) 

Research to determine the extent to which the study results (student 
behavior influences teacher behavior in predictable directions) 
generalize to other non-education courses with regular (not guest) 
teachers. Consider research in which input variables, such ass (1) 
student and teacher characteristics which the individual possess before 
entering the class, (2) verbal and non-verbal cognitive and affective 
student and teacher classroom interactions, and (3) student and teacher 
product measured after a designated interval. (2030/1971) 
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E. Investigate the Effects of the Learning Envir»3nment 



Replication of the entire series of studies on classroom climate and 

individual learning, with revised and more reliable instruments and using 
a national random sample. (1409/1968) 

Re3earch to identify experimentally manipulated factors and status 
variables which affect the social environment. (1306/1969) 

Study of the environment, traced on individual learners grouped by mental 
ability, biographical characteristics, and personality. (1306/1969) 

Pilot testing to determine at what point, if any, the observer is forgotten 
and the "natural" classroom situation emerges again. (1326/1969) 

Research into person-environment interactions on college campuses. (2222/ 
1969) 

Research to answer this question: Can the class composition be manipulated 
with respect to the biographical characteristics of its members to 
achieve an optimal learning environment for each student? (1026/1970) 

Research to examine the dynamics and behavior of students as they interact 
and perform in two types of college environments — "impersonal" and 
"supportive." The following hypotheses are offered: (1) a highly 
independent student would do equally well in either environment, (2) 
a dependent student would do best on a "supportive" campus, and (3) 
a dependent student of high ability would perform better at an "impersonal" 
campus than would a dependent student of low ability. (1030/1970) 

Research on the following questions: What are the more critical student 
activities that affect what is learned in the more important in- 
stitutional settings? What are the factors that modify mathemagenic 
activities? What causes positive mathemagenic activities to deteriorate? 
How can one give a coherent account of Class III activities that is 
consistent with what is known about visual perception, memory, and 
language processes? (1050/1970) 

Determination of the efficacy of peer influence and vicarious reinforce- ^ 
ment procedures in effecting behavior change in a classroom setting. 
(764/1971) 

Research to determine whether the effects of the physical environment wash 
out when the error variance is set up to be carefully observed. There 
may be more consistency in response within Ss than within rooms. (1668/ 
1971) 

Study of the effects of a complete honor system, and the trust implied by 
such an honor system, on the attitudes, values, and behaviors of various 
kinds of students. It may then be possible to delineate the k{nd of 
educational atmosphere most conducive to the growth and strengthening 
of academic responsibility and intellectual values. (2032/1971) 
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